System and method for biometric identification

ABSTRACT

The present invention relates to a method for generating a biometric signature of a subject comprising: obtaining a plurality of sequential video frame images of a moving subject from a video segment; obtaining a portion of each frame comprising a surrounding of the moving subject; carrying out a transformation function to the frequency domain on one or more of said portions of the frames comprising a surrounding of a of the subject; and optionally saving the spectral characteristics of said transformation function in a repository. The present invention also relates to a system for carrying out said method.

FIELD OF THE INVENTION

The present invention relates to the field of biometric identificationthrough image and signal processing. More particularly, the presentinvention relates to the identification of a person by spectrumanalyzing of a moving person's surrounding.

BACKGROUND OF THE INVENTION

Current biometric methods for the image identification of a subject arebased on clear facial, iris, handprints etc., and require specialequipment and clear photographs from specially installed cameras.Several biometric methods are ineffective when used with standardsecurity cameras since they have a relatively low resolution, they areplaced at generally high angles and function with uncontrolled lightingconditions. One of the outcomes of these drawbacks is that the peopleidentification from these cameras are inefficient. Today, trackingmethods are based on the fact that a camera can track certain objectsuntil the objects exit that camera's field of view. When a person exitsa certain camera's field of view and enters an adjacent camera's fieldof view the tracking of the first camera is ceased and a new trackingbegins by the second camera. The tracking of the second camera isimplemented independently, regardless of the first camera tracking.Automatic continuous tracking of a certain object from one camera'sfield of view to another camera's field of view includes complicatedtracking applications which are inaccurate and frequently tend tomalfunction. Furthermore, a tracking method which enables tracking evenwhen a subject exits all the camera fields of view (or is obscured byanother object) and returns later on, is highly needed.

Furthermore, there is a need for tracking the whereabouts of a personwhen analyzing a post event video and looking for the timeline of eventsregarding a specific person. A common solution today is to have asecurity analyst view the video and mark manually the appearance of acertain individual.

FIG. 1 illustrates a prior art series of cameras (50 a-50 e) aiming atcovering the surrounding field of view of a warehouse. Each cameracovers a certain field of view. Each camera's adjacent camera covers afield of view adjacent to its camera's field of view. Security personnelviewing the camera filming recording at a remote location would havedifficulties tracking a suspicious subject when the suspicious subjectcrosses one camera's field of view to another. Current systems allowmarking a subject on the camera viewing screen. Then the subject istracked using appropriate applications until the subject exits thecamera's field of view. The security personnel would have to mark thesuspicious subject again on the screen of the adjacent camera forcontinuation tracking, what could be very confusing due to the fact thatpeople look alike on a security camera. Furthermore, constant trackingalong a series of cameras requires frequent manual interference.

Also, means are required for tracking and identifying a subject even ifhe exits all system cameras field of view for a long period of time.

US20110194762 relates to a method for detecting a hair region, includesacquiring a confidence image of a head region; and detecting the hairregion by processing the acquired confidence image. The hair regiondetection method may detect the hair region by combining skin color,hair color, frequency, and depth information.

WO2014/203248 provides certain solutions to the aforementioned problems.This publication relates to a method and system for generating andcomparing a biometric singular signature of a person comprising thesteps of a) obtaining a first image of a person; b) obtaining a hairportion image of the person; c) transforming the hair portion image intoits frequency domain image and optionally saving said frequency domainimage in a database. However, the detection process in this publicationnecessitates obtaining an image with a clear portion of a person's hairand/or head contour in order to carry out the detection.

It is therefore an object of the present invention to provide a methodand means for coherent identification of a person with a novel biometricquality based on his effect on the surroundings.

It is yet another object of the present invention to provide a methodand means for generating a digital signature accordingly.

It is yet another object of the present invention to provide a methodand means for performing a signature on a subject and means foridentifying the signed subject later on when returning to system camerasfields of view.

It is yet another object of the present invention to provide means toanalyze a post event video to determine the whereabouts of a specificperson during the Video run time.

It is yet another object of the present invention to generate asignature for a person from a first video sequence and search for thatspecific person in a second video sequence, generated at a differenttime, on-line or post event analysis.

Other objects and advantages of the present invention will becomeapparent as the description proceeds.

SUMMARY OF THE INVENTION

The present invention relates to a method and system for obtaining abiometric identification of the frequency spectral characteristics of asurrounding area of a moving subject person. The preset inventioncomprises obtaining a sequence of video frames and performing imageprocessing to the frames. The frequency spectral characteristics of asurrounding frame portion of the moving person is obtained and may bestored in a database and compared with second frequency spectralcharacteristics of a surrounding area of a second subject person so asto determine whether an identification therebetween is held positiveindicating that the first and second subject persons are actually thesame person. The comparison of both frequency spectral characteristicsis such that if the coherence level therebetween is above a predefinedthreshold—the determination is deemed positive.

The present invention comprises generating a vector according to themovement of the subject person and determining an area—Region OfInterest (ROI)—at a distant position in relation to the generatedvector. The ROI is integrated at corresponding locations on each of thesequence frames and a transformation function to the frequency domain isapplied thereon. The frequency spectral characteristics obtainedtherefrom are optionally stored in a database.

The present invention relates to a method for generating a biometricsignature of a subject comprising:

obtaining a plurality of sequential video frame images of a movingsubject from a video segment;

obtaining a portion of each frame comprising a surrounding of the movingsubject;

carrying out a transformation function to the frequency domain on one ormore of said portions of the frames comprising a surrounding of thesubject; and

optionally saving the spectral characteristics of said transformationfunction in a repository.

Preferably, obtaining a portion of each frame comprising a surroundingof the moving subject comprises:

obtaining foreground frames corresponding to the plurality of sequentialvideo frames, each comprising the moving subject;

generating a vector representing the direction of the moving subject;

determining a Region Of Interest (ROI) at a position in relation to saidvector; and

determining said portion of each frame comprising a surrounding of themoving subject, at a location on each frame corresponding to thelocation of said determined ROI.

Preferably, obtaining foreground frames corresponding to the pluralityof sequential video frames comprises:

obtaining a background frame comprising the background of the pluralityof sequential video frame images;

subtracting said background frame from each of the plurality ofsequential video frames.

Preferably, generating a vector representing the direction of the movingsubject comprises:

obtaining body portions of the foreground objects of the foregroundframes;

obtaining reference point frames corresponding to the foreground frames,each comprising a reference point about or at an edge of thecorresponding location of said body portions;

combining all the reference points frames into a three-dimensionalcoordinate system frame comprising all the reference points being attheir corresponding location on the three-dimensional coordinate systemframe;

determining a vector in the three-dimensional coordinate system frameaccording to a sequence of a number of reference points from thereference points, that produce the most stable vector.

Preferably, determining a Region Of Interest (ROI) at a position inrelation to said vector comprises:

-   -   a. obtaining a plurality of background sequential video frames        being a sequence of frames comprising the background of the        plurality of sequential video frame images;    -   b. integrating the vector determined in in each background        frame;    -   c. determining an initial ROI on each background frame being at        a predetermined corresponding position from said vector;    -   d. carrying out a transformation function to the frequency        domain in time to the ROI portions of said background frames        obtaining spectral characteristics and determining the stability        frequencies from said spectral characteristics;    -   e. integrating each of the plurality of sequential video frame        images with said initial ROI determined and carrying out a        transformation function to the frequency domain in time to the        initial ROI portion of said sequential video frames thus        obtaining spectral characteristics and storing the sensitivity        of said stability frequencies of said spectral characteristics        of the initial ROI;    -   f. shifting the initial ROI to a surrounding area in each of the        plurality of sequential video frame images, and carrying out a        transformation function to the frequency domain in time to the        currently shifted ROI portion of said sequential video frames        thus obtaining spectral characteristics and storing the        sensitivity of said stability frequencies of said spectral        characteristics of the currently shifted ROI;    -   g. repeating step f according to a predetermined shifting rule;    -   h. after step f has been repeated for all sequences of the        shifting rule determining the ROI of the initial and shifted        ROIs with the highest sensitivity stored as the ROI.

Preferably, the body portions are one or more of:

the head portion;

the center body portion;

the feet portion.

Preferably, the method further comprises a step of identification bycomparing the obtained spectral characteristics of the transformationfunction with spectral characteristics saved in a database, wherein anidentification result is deemed to be positive when the coherence levelbetween both compared frequency spectral characteristics is above acertain threshold.

Preferably, the method comprises performing a signature for a subject byobtaining and saving the vector generated and the corresponding ROIportion determined, and further obtaining and saving one or more of thefollowing items in a database in relation to said subject:

the spectral characteristics of a transformation function to thefrequency domain in time to the ROI portions of the sequential frames;

the spectral characteristics of a transformation function to thefrequency domain in time to the ROI portions of the background frames;

the spatial spectral characteristics of a transformation function to thefrequency domain at the ROI portion of one of the sequential frames;

the spatial spectral characteristics of a transformation function to thefrequency domain at the ROI portion of one of the background frames.

Preferably, the method further comprises a step of identification;

providing a signature of a subject person stored in a data basecomprising a vector, an ROI, spectral characteristics of atransformation function to the frequency domain in time of a framesequence, spectral characteristics of a transformation function to thefrequency domain in time of background frames;

wherein the Region Of Interest (ROI) at a position in relation to thevector is determined such that it is in the same position in relation tothe vector as the signature ROI in relation to the signature vector;

wherein said method further comprises:

i. obtaining the spectral characteristics of a transformation functionto the frequency domain in time to the ROI portions of the backgroundframes;

ii. obtaining a relative difference by inputting the spectralcharacteristics of a transformation function to the frequency domain intime of the background frames of step (i) and said signature spectralcharacteristics of a transformation function to the frequency domain intime of background frames, into a transfer function;

iii. obtaining the spectral characteristics of a transformation functionto the frequency domain in time to the ROI portions of the sequenceframes and shifting the value of said spectral characteristics of atransformation function to the frequency domain in time to the ROIportions of the sequence frames, in a proportional manner as saidrelative difference;

iv. comparing the shifted values to the signature spectralcharacteristics of a transformation function to the frequency domain intime of a frame sequence;

wherein an identification result is deemed to be positive when thecoherence level between the compared spectral characteristics of step(iv) is above a predefined threshold.

Preferably, the method further comprises an identification, said methodfurther comprising:

obtaining an error value between the spectral characteristics of thetransformation function and spectral characteristics saved in adatabase;

wherein an identification is deemed positive if one of the followingconditions are met:

I) the error value is beneath a predefined threshold value; II) thefollowing consecutive steps are carried out less times than apredetermined threshold number:

-   -   i. transferring the error value to an adaptive filter that        adapts the values of one of the spectral characteristics        according to the error value;    -   ii. obtaining an error value between the adapted spectral        characteristics value and the other spectral characteristics;    -   iii. determining if the error value of step (ii) is beneath said        threshold value;    -   iv. returning to step (i) when the determination of the error        value of step (iii) is deemed negative.

Preferably, the method further comprises an identification, said methodfurther comprising:

obtaining an error value between the spectral characteristics of thetransformation function and spectral characteristics saved in adatabase;

wherein an identification is deemed positive if one of the followingconditions are met:

I) the error value is beneath a predefined threshold value;

II) the following consecutive steps with possible recursion are fullycarried out during a time duration less than a predefined thresholdtime:

-   -   i. transferring the error value to an adaptive filter that        adapts the values of one of the spectral characteristics        according to the error value;    -   ii. obtaining an error value between the adapted spectral        characteristics value and the other spectral characteristics;    -   iii. determining if the error value of step (ii) is beneath said        threshold value;    -   iv. returning to step (i) when the determination of the error        value of step (iii) is deemed negative.

The present invention relate to a system comprising one or more camerasconnected to processing means, wherein the processing means comprise:

A) a database;

B) a transformation to frequency domain module;

C) a comparing frequency coherence function module.

The present invention system processing means is configured to carry outall the method steps described herein (e.g. method steps relating totransformation functions, comparison functions, filtering functions,shifting functions, error functions, absolute value functions, luminancefunctions, convolution functions, contrast adjusting functions,optimization functions, stable frequency reading functions, frequencysensitivity reading functions, transfer functions, coherence comparisonfunctions, etc.).

The present invention relates to a system comprising one or more camerasconnected to processing means, wherein the processing means comprise:

A) a database;

B) a transformation to frequency domain module;

C) a comparing frequency coherence function module; wherein theprocessing means are configured to generate a biometric signature of asubject comprising the steps of:

obtaining a plurality of sequential video frame images of a movingsubject from a video segment;

obtaining a portion of each frame comprising a surrounding of the movingsubject;

carrying out a transformation function to the frequency domain on one ormore of said portions of the frames comprising a surrounding of thesubject; and

optionally saving the spectral characteristics of said transformationfunction in a repository.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example in theaccompanying drawings, in which similar references consistently indicatesimilar elements and in which:

FIG. 1 illustrates a prior art system.

FIG. 2 illustrates an embodiment of the system of the present invention.

FIGS. 3A-3C illustrate method steps to obtain a background frameaccording to an embodiment of the present invention.

FIGS. 4A-4B illustrate method steps to obtain foreground framesaccording to an embodiment of the present invention.

FIGS. 4C-4D illustrate using an example filter according to anembodiment of the present invention.

FIG. 5 illustrates method steps to obtain reference point framesaccording to an embodiment of the present invention.

FIG. 6 illustrates method steps to obtain vectors according to anembodiment of the present invention.

FIG. 7 illustrates method steps to obtain regions Of Interest (ROIs)according to an embodiment of the present invention.

FIG. 8 illustrates an example of a signature according to an embodimentof the present invention.

FIGS. 9A-9B illustrate the vector and ROI on a sequence frame accordingto an embodiment of the present invention.

FIG. 10 illustrates a transfer function and shifting according to anembodiment of the present invention.

FIG. 11 illustrates a coherence comparison function according to anembodiment of the present invention.

FIG. 12 illustrates a coherence comparison function with filteraccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Several specific details are provided herein, such as examples ofdevices/components, and methods, to provide a thorough understanding ofembodiments of the present invention. A person skilled in the art willunderstand, however, that the present invention can be implementedwithout one or more of the specific details or alternatively, well-knowndetails are not described for the sake of clarity (and would be clearlyunderstood by a person skilled in the art).

Some components disclosed herein may be implemented in hardware,software, or a combination thereof (e.g., firmware). Software componentsmay be in the form of computer-readable program code stored in acomputer-readable storage medium (e.g. a memory, mass storage device,removable storage device). For example, a computer-readable storagemedium may comprise computer-readable program code for performing thefunction of a particular component. Likewise, computer memory may beconfigured to include one or more components, which may be executed by aprocessor. Software components may be implemented in logic circuits, forexample. Components may be implemented separately in multiple modules ortogether in a single module.

The present invention relates to a method and system for biometricidentification of a person by processing the interaction between personsand their surroundings. The present invention comprises obtaining aplurality of sequential video frame images of a person typically moving,obtaining portions of the frames comprising the surrounding of theperson in motion and performing spectrum analysis in the frequency rangeof those portions. The processing of the video frames concludes with asignature of the analysis product and saving the same in a repository.The saved signature may be used for future comparison and recognitionverification with another analyzed plurality of video frames, even froma different camera and/or different background. If the other analyzedplurality of video frames is similar to the first (with a level ofcoherence above a certain threshold in the frequency domain) then apositive identification is determined.

According to one implementation, the present invention relates to asystem comprising one or more video cameras, such as standard securitycameras (e.g. standard security video cameras). FIG. 2 illustrates anembodiment of the invention, wherein a series of cameras (50 a-50 e) areplaced on top of a building aiming at covering the surrounding of abuilding (e.g. a warehouse building). Each camera covers a certain fieldof view adjacent to the adjacent camera's field of view. Securitypersonnel can view the cameras filming recordings at a remote location.The system enables tracking capabilities and allows security personnelto mark a subject on the camera viewing screen for tracking saidsubject, using appropriate tracking applications such as “Six Sense” byNESS technologies, or such as MATLAB tracking application.

The one or more cameras 50 a-50 e are connected to (or wirelessly inconnection with) processing means 55 such as a standard computer. Theprocessing means 55 are adapted to take a plurality of sequential videoframe images (“video frames” or just “frames” used hereininterchangeably) of a subject and to analyze a specific region of itssurrounding in the frequency domain. The region in the frequency domainis then marked with a signature (representing the subject) and stored ina database. The system provides the ability such that when the subjectenters the field of view of another camera, or disappears and returns tothe same camera field of view, the subject new surrounding is analyzed,(optionally signatured) and compared with the system signature database.The system matches between the new measured properties and thesignatures stored in the database. If the coherence between the compareditems is above a certain threshold (i.e. matching items) then theidentification is deemed positive, thus determining that the subjectrepresented by the currently obtained analysis is the subjectrepresented by the matching signature. Thus system personnel areinformed of the positive identification.

According to a preferred embodiment of the present invention, theanalysis of the frames method and the signature are implementedaccording to one or more of the following:

Obtaining the Background of the Frames

A plurality of sequential digital image frames 10 are extracted from adigital video of a system filming recording camera and are saved in asystem database and are analyzed/processed by the processing means 55(e.g. a processor interactive with a memory and configured to carry outfunctions of various function modules) coupled thereto. The framesextracted typically comprise a moving subject person. At least some ofthe frames 10 comprise at least portions of the moving person. Theframes comprise a foreground which relates to the moving objects withinthe image such as the moving person, and a background at the areas whichare not part of the moving objects foreground. The number of theplurality of frames 10 is usually between 4 and 100 frames.

It should be noted that the present invention is especially useful witha surveillance video camera and the frames are extracted when the camerais active in a still position such that the general background regionslocation in the frames (the portions without the moving subject person)remain similar. However, it should be noted that the present inventionmay be used with a still camera or a movable camera and with sequencesof a moving camera as well.

As a first part of the processing, each one of the plurality of frames10 is processed such that three sets of frames (two additional sets) areobtained for further processing:

(1) A frame without the moving object, being background frame 15 (or15′).

(2) A set of frames with only the moving object being foreground frames20 (or 20′).

(3) A set of frames with the moving object (foreground) and thebackground—10 (typically the original extracted frames 10 or framescorresponding to the original frames with possible additional processingthat aid future calculations).

According to a preferred embodiment of the present invention, abackground frame 15 is found by running the recording video back orforward for a predetermined time, such that a frame 15 will be obtainedwithout (or with a minimal portion of) the moving subject person (eitherbefore he enters the camera region or after he exits the camera region).An example for obtaining the background frame comprises the systemobtaining a short video portion at a certain time before or after (ortowards the beginning or towards end of) the analyzed sequence (e.g. afew seconds before or after the sequence) and subtracting one of theframes from the short video portion from another frame of the shortvideo portion (e.g. subtracting the last frame image from the firstframe image of the short video portion). If the result is close to zero(null image) then one of said 2 images in the subtraction process isdetermined as the background frame 15.

According to another embodiment the background frame 15′ is obtained asfollows. To assist calculation, the processing means comprise a buffer11 which transfers each of the 2D-signal frames 10 into a 1-D signal, asshown in FIG. 3A. An illustrative example of the pixels of the 2-D stillimage frames representation and the 1-D representation can be seen inFIGS. 3B and 3C respectively.

The processing means comprise an image statistics function module 12(e.g. 2-D median function module), which takes the output frames (fromthe buffer 11) 1-D signals (representing the frames 10) and performs amedian function on them, thus practically removing the moving objectfeatures (moving person) from each of the 1-D signals and remaining withthe backgrounds of the frames. According to one embodiment, the medianfunction includes finding a median threshold of the intensity values ofthe pixels of the image references set of frames 10. The signals in thebackground portions of the frame images are almost identical. Afterperforming the median function, generally, the pixels with intensityvalues beneath the threshold are considered the background portion. Thepixels with intensity values above the threshold are considered theforeground portions (portions with the moving subject person). The pixelareas of the foregrounds of the images frames are assigned with thecorresponding other background image values (at correspondinglocations). Preferably, in case of RGB images, the intensity is thevalue of the total amplitude of each pixel.

According to an embodiment of the present invention, the medianthreshold is the numerical value separating the higher half of a datasample from the lower half. For example, count(n) is the total number ofobservation items in given data. If n is odd then—

Median (M)=value of ((n+1)/2)th item term.

If n is even then—

Median (M)=value of [((n)/2) th item term+((n)/2+1) th item term]2.

Example

For an Odd Number of Values:

As an example, the sample median for the following set of observationsis calculated: 1, 5, 2, 8, 7.

Firstly, the values are sorted: 1, 2, 5, 7, 8.

In this case, the median is 5 since it is the middle observation in theordered list.

The median is the ((n+1)/2)th item, where n is the number of values. Forexample, for the list {1, 2, 5, 7, 8}, n is equal to 5, so the median isthe ((5+1)/2)th item.

median=(6/2)th item

median=3rd item

median=5

For an Even Number of Values:

As an example, the sample median for the following set of observationsare calculated: 1, 6, 2, 8, 7, 2.

Firstly, the values are sorted: 1, 2, 2, 6, 7, 8.

In this case, the arithmetic mean of the two middlemost terms is(2+6)/2=4. Therefore, the median is 4 since it is the arithmetic mean ofthe middle observations in the ordered list.

We also use this formula MEDIAN={(n+1)/2}th item. n=number of values

As above example 1, 2, 2, 6, 7, 8; n=6; Median={(6+1)/2}th item=3.5thitem. In this case, the median is average of the 3rd number and the nextone (the fourth number). The median is (2+6)/2 which is 4.

If A is a matrix, median(A) treats the columns of A as vectors,returning a row vector of median values.

The still background image outputted from the image statistics buffer 12is transferred to a reshaping module 13 along with the original imagesize of the frames 10 thus re-producing a complete 2-D background frame15′.

Obtaining the Foreground of the Frames

A luminance normalization is applied to the original frames such thatthe frames 10 luminance is adjusted to the luminance of the backgroundframe 15 (or 15′, for simplicity we will refer only to 15), as shown inFIG. 4A. The processing means comprise a luminance normalization module14 which is adapted to change the luminance of one input image to thatof another input image. The original frames 10 and the background frame15 are transferred to the luminance normalization module 14, whichadjusts the luminance of each of the original frames 10 to that of thebackground frame 15. Each luminesced output frame 16 of the luminancenormalization module 14 is subtracted from the background frame 15 by asubtracting module 17 a. The processing means comprise an absolute valuefunction module which produces the absolute value of an input imageframe. The result of the subtraction 17 b (the output of subtractingmodule 17 a) is transferred to the absolute value function module 18 andproduces the absolute value of subtraction 17 b. Consequently,corresponding object foreground frames 20 (outputted from absolute valuefunction module 18) are obtained comprising the objects (e.g. movingpeople) of the original image.

Optionally, an improved object foreground frames 20′ can be obtainedcomprising improved objects (e.g. moving people) of the original image,as shown in FIG. 4B. The object foreground frames 20 are transferred tobuffer 6 which transfers them into a 1-D signal (similar to the buffer11 function). The 1-D signals are transferred to FIR filter 7 whichfurther filters noises of background portions. The mathematics filterimplementation is like a classic FIR convolution filter:

${y(k)} = {\sum\limits_{n}^{\;}{{u\left( {n - k} \right)}*{h(k)}}}$

Wherein y—is the output signal, u is the input signal, h is the filter(array of coefficients, such as a Sobel operator filter), k is thefilter element (of the array of coefficients), and n is an index numberof a pixel. k and n are incremented by 1.

The filtered frames are then transferred to a reshaping module 8 alongwith the frame size of frames 20 thus re-producing a complete 2-Dimproved foreground frames 20′.

FIGS. 4C and 4D show an example of an image 120 before the filtering,and of the image 120′ after the filtering. It is clear that backgroundportions (e.g. ground portion 110) that appear in FIG. 4C do not appearin FIG. 4D.

Obtaining the Body Portions of the Foreground Objects

The next stage comprises obtaining the body portions of a person objectin the frames. Obtaining the body portion can be done by known functionsin the art. One manner of obtaining the body portion is using suitablefilters/templates such as Wavelet templates to be applied on the image.Optionally, an average of a group of Wavelet templates can be used.

The processing means comprise a contrast adjusting module which adjuststhe contrast of an image such that it becomes optimal as known in theart (shown in FIG. 5). The Contrast Adjustment module 22 adjusts thecontrast of an image by linearly scaling the pixel values between upperand lower limits. Pixel intensity values that are above or below thisrange are saturated to the upper or lower limit value, respectively. Thecontrast adjustment module provides the system with improvedidentification results.

The foreground frames (20 or 20′) are preferably transferred to thecontrast adjusting module 22 (comprised in the processing means) whichoptimizes the frames contrast.

A wavelet template is used in the method for obtaining body portionswithin the images. The output of the contrast adjusting is transferredto a FIR convolution module 23 comprised in the processing means. TheFIR convolution module 23 convolves the contrast adjusted frames with aselected Wavelet body portion template 19 (or other filter template) ina FIR manner (similarly as explained hereinabove) producingcorresponding frames 24 each having an additional coefficients matrixdimension, wherein each frame pixel has a corresponding coefficient ofsaid matrix.

The mathematics filter implementation is like classic FIRconvolution-decimation filters:

${y(k)} = {\sum\limits_{n}^{\;}{{u\left( {n - k} \right)}*{h(k)}}}$

Wherein y—is the output signal, u—input signal, h—is the filtercoefficients, k, n—indexes where the index k is incremented by 1, andindex n—is incremented by Decimation factor, which is changing from 1 to2{circumflex over ( )}(Wavelet Levels Number).

The portions of the image with high coefficients (in the additionalcoefficients matrix dimension) are the body portions (having a bodyshape) of the foreground objects. The high coefficients are produced dueto the compliance of the foreground image body portions and the template19 characteristics.

Obtaining Direction Vectors

A Local pre-defined points function module 25 (comprised in theprocessing means) applies a function on the obtained convolved outputframes 24 (output of module 23) to obtain corresponding frames 26 withthree reference points on each frame. One reference point on each frameis located at the corresponding location to that of the head of theforeground image body portion (preferably top of head). The secondreference point on each frame is located at the corresponding locationto that of the center of the body of the foreground image body portion.The third reference point on each frame is located at the correspondinglocation to that of the bottom of the feet of the foreground image bodyportion. The function that locates these three points is based on BlobAnalysis or other similar image processing methods. In cases whereframes have only portions of the body (e.g. only head and center body oronly feet, etc.) then only the existing corresponding points will befurther analyzed.

The processing means comprise a 3D transformation module 30. Thereference points on each of the frames 26 in the sequence aretransferred to a 3D transformation module 30, as shown in FIG. 6. The 3Dtransformation module 30 combines all the reference points frames 26into a three-dimensional coordinate system comprising all the referencepoints being at their corresponding location on the three-dimensionalcoordinate system. The three-dimensional coordinate system correspondsto one frame (having the similar frame size as frames 26) with all ofthe reference points appearing on it at their corresponding 3D location.It should be noted that the sequence order of the reference points isalso stored.

A sequence analysis module 31 analyses the plurality of reference pointsof the “head” reference points on the three-dimensional coordinatesystem, and searches for a sequence of a predetermined number of “head”reference points (preferably between 4-10) along the whole “head” pointssequence, that produces the most stable vector—Vec_h (comprised of thepredetermined number of “head” points). Each “head” reference point onthe three-dimensional coordinate system has a 3D location (on the xaxis, y axis and z axis) and the most stable vector, i.e. the predefinednumber of sequential “head” reference points that are most linear, aredetermined to be the most stable vector Vec_h.

The sequence analysis module 31 analyses the plurality of referencepoints of the “body center” reference points in the same manner mutatismutandis producing the most optimal center body vector Vec_cb. The 3Dtransformation module 30 analyses the plurality of reference points ofthe “feet” reference points in the same manner mutatis mutandisproducing the most optimal feet vector Vec_f.

Obtaining ROIs

Certain areas near the vectors (when integrated in certain frames) areobtained for spectrum analysis, herein referred to as ROIs (regions ofinterest). These ROIs are preferably on surfaces near the vectors (e.g.walls, floor, ground) such that the moving subject person has a maximaland selective effect on the spectral frequency response (and withminimal noise) at the ROI locations. The dispersing of the spectrum ofthe light energy measured there is clear and specific (sharp and notsmeared) and all this in relation to a location where the subject personis not present. The ROIs are determined according to the optimal lightreflex and best resolution in the frequency band. Thus the ROIs arespectrally analyzed.

According to one embodiment of the present invention, a sequenceplurality of background frames 10 bg are obtained by a short videoportion at a certain time before or after (or towards the beginning ortowards the end of) the analyzed sequence frames 10 (e.g. a few secondsbefore or after the sequence). For determining them as backgroundframes, for example, each frame of the potential background frames inthe sequence is subtracted from one specific background frame of thesequence (e.g. the most middle one). If the subtraction result is closeto zero (null image, up to a certain level) for each subtraction thenthat sequence of frames is indeed determined as a background frames 10bg sequence. Otherwise, another sequence is checked (e.g. a few secondsbefore/after), and so on and so forth, until all the frames in thepotential background sequence of frames are determined as a backgroundframes 10 bg sequence.

Then, the vectors Vec_h, Vec_cb and Vec_f are integrated with thebackground frame sequence 10 bg such that each of the background frames10 bg now comprises a three-dimensional coordinate system, with saidvectors at their corresponding locations (i.e. at the same relativeareas in the frame e.g. same relative pixels).

The ROIs may be various shapes e.g. squares, rectangles, circles, on thebackground frames 10 bg and various sizes. The ROIs may be found inseveral manners. Initially, a point on the background frames 10 bg beingat a predefined distance at 90° (or other predefined angle) from headvector Vec_h center is defined as the center of the ROI. It should benoted that the ROI preferably has a two-dimensional shape but itsposition is in the three-dimensional coordinate system initiallyparallel to the head vector Vec_h. Typically the ROI near the headvector Vec_h is most efficient when on a wall. Optionally, 3D knowntechniques may be used for identifying a wall (or ground) and thedistance of the wall from the vector. An appropriate distance of theinitial ROI may be determined accordingly.

A transformation function to the frequency domain is applied (e.g. FFT)on the relevant time axis to the initial ROI locations of the backgroundframes 10 bg, thus producing the spectral characteristics, e.g. PSD(power spectral density and/or PeakToPeak, PeakToRMS, RMS, etc.) of thatROI indicating its amount of energy. Particularly, the most stablefrequencies, i.e. the frequencies which change the least (or do notchange at all) over the measured time produced in the transformationfunction are stored for further analysis.

The initial ROI is integrated with the plurality of sequential digitalimage frames 10 such that each of the frames 10 now comprises athree-dimensional coordinate system, with said initial ROI at itscorresponding locations. The signals (image portions) of each of theframes 10 at the corresponding locations to the initial ROI areobtained. The processor carries out a transformation to the frequencydomain (e.g. FFT) function in time to all the signals in the frames 10corresponding locations to those of the initial ROI thus obtainingspectral characteristics (e.g. PSD) values in the frequency range forthe initial ROI.

The present invention comprises an optimization process 40. Theoptimization process comprises ROIs being integrated with the pluralityof sequential digital image frames 10 shifting the ROI along threedimensional axes in predefined increments on all three axes (andcombinations thereof). In each shift, the transformation function intime is applied to the ROI currently being checked at its correspondinglocation in the sequence frames 10. In each shift the sensitivity of thestable frequencies (found in the transformation function of the initialROI of the background frames 10 bg) are evaluated by the processorperforming a transformation to frequency domain function (e.g. FFT) intime to all the signals in the frames 10 corresponding locations tothose of the ROI being checked thus obtaining spectral characteristics(e.g. PSD) values in the frequency range for the ROI being checked.After checking the sensitivity of the stable (stability) frequencies inall of the potential ROIs (all of the ROIs checked during the shiftingprocess) the ROI with the highest sensitivity (i.e. with the stablefrequencies that are mostly changed relative to their characteristics inthe background frames 10 bg transformation function)—ROI_h, isdetermined for further analysis. Furthermore, the spectral properties(e.g. PSD) of frames 10 bg are calculated by a transformation functionin time at regions ROI_h.

An example of the shifting is as follows. The center point of theinitial ROI is considered as the center of a three dimensionalcoordinate system (0,0,0) wherein the head vector Vec_h is parallel tothe X axis (e.g. first Cartesian axis), the center of the vector andcenter (0,0,0) are on the Y axis (e.g. second Cartesian axis) and theperpendicular axis to the X and Y axes (e.g. third Cartesian axis) isthe Z axis. The center point of the initial ROI is shifted such that itwill pass on all whole points (with whole numbers) of a cuboid havingcenter at (0,0,0) wherein each predefined increment is a whole number.The shifting is carried out according to a shifting rule until all theshifted ROIs according to the rule are evaluated. Other shifting methodsas known in the art may also be used such as shifting according to a3-dimensional Polar coordinate system e.g. varying distance from centerof vector Vec_h and varying angles therefrom, etc. Other shiftingmethods includes semi-random shifting, etc. Optionally, if a sensitivityevaluation of a certain ROI being checked is found to be good (above acertain threshold) then the corresponding ROI may be determinedimmediately as ROI_h without continuing the rest of the optimizationprocess according to the shifting rule.

FIGS. 9A-9B show an illustrative example in order to understand theinvention more clearly and to obtain a more illustrative feeling of theinvention. FIG. 9A illustrates one of the sequence frames 10 comprisinga walking person. FIG. 9B illustrates the head vector Vec_h and theinitial ROI In_ROI_h (and the distance therebetween 5) integrated in thesequence frame 10 of FIG. 9A. Furthermore, for illustrative purposes thehead reference points 4 of all the sequence frames which generate thevector Vec_h are shown along the vector (for illustrative purposes andare not necessarily geometrically exact).

The optimization has been explained hereinabove with relation to thehead vector Vec_h determining the ROI with the highest sensitivityROI_h. Appropriate ROIs with the highest sensitivity are found inrelation to vectors Vec_cb and Vec_f after a similar optimizationprocess, in the same manner mutatis mutandis (for simplicity the wholeprocess has not been written again for Vec_cb and Vec_f). Thus the ROIswith the highest sensitivity—ROI_cb (in relation to center body vectorVec_cb) and ROI_f (in relation to feet vector Vec_f) are determined forfurther analysis (shown in FIG. 7).

Optionally the other sides of the vectors (90° of the other side, or270°) can also be checked and optimized in a similar manner obtainingthe best ROI result of the two sides to be the ROI for furtherprocessing.

It should be noted that typically the ROIs near Vec_cb and Vec_f aremost efficient when on the floor/ground.

It should be noted that according to one specific embodiment, in caseswhere a sequence of “pure” background frames are hard to find (e.g. whenmost appropriate to use the image statistics function module 12 asexplained hereinabove e.g. 2-D median function module) a sequence ofbackground frames 10 bg are used where the subtraction from one anotheris minimal (even if not close to zero). According to one embodiment, theROI is shifted only within the locations that have been found to bebackground locations during the “Foreground/Background separationprocess as explained hereinabove (with the image statistics functionmodule 12, 2-D median function module, etc.).

Furthermore, the initial ROI integrated with the plurality of sequentialdigital image frames 10, may be determined also by an optimizationprocess (and not necessarily by a predefined area in relation to itscorresponding vector). According to this embodiment, a first initial ROIis determined according to a predefined area in relation to itscorresponding vector. Then the first initial ROI is shifted (in asimilar manner according to one of the methods explained hereinavove)and the stable frequencies are measured. The result of the stability ofeach of the shifted ROIs are evaluated and the one with the higheststability is determined as the (initial) ROI for the subsequentoptimization stage according to the sensitivity (in relation to saidhighest stability). Then the final ROI determined is the one with thehighest sensitivity, as explained hereinabove, etc.

Signature

The term “signature”, or “signatured” or “signed” (in past tense) referto saving a frequency factor in the processing means database under acertain name/identification.

The spectral characteristics (e.g. PSD) values in the frequency rangefor the determined ROI with the highest sensitivity (of the stabilityfrequencies) ROI_h is stored in the system memory/database. This storingis actually part of the signature of the subject person saving hisbiometric characteristics based on quantum radio physics of the subjectperson in the system memory/database.

The same thing is carried out in relation to the spectralcharacteristics (e.g. PSD) values in the frequency range with thehighest sensitivity (of the stability frequencies) ROI_cb and ROI_f(near the center body and feet respectively). They are also storedmutatis mutandis.

Optionally, the processor carries out a spatial transformation tofrequency range function (e.g. FFT) to one or more of the sequenceframes 10 and to one or more of the background frames 10 bg at thecorresponding ROI_h, ROI_cb and ROI_f locations thus obtaining spectralcharacteristics (e.g. PSD) values of spatial features at thecorresponding ROI_h, ROI_cb and ROI_f locations. According to thisoption they are also stored as part of the signature. Typically, whencarrying out the transformation and saving in the spatial range for onlyone frame of the sequence frames 10 it can be the middle frame in thesequence. Typically, when carrying out the transformation and saving inthe spatial range for only one background frame of the sequencebackground frames 10 bg it can be the middle frame in the sequence.

FIG. 8 shows an embodiment of the present invention—a full signature 50comprising 12 frequency spectral characteristics (for example the PSD)results stored as representing the subject person. Furthermore, thevectors and optimized ROIs (optimized ROI locations) leading to thespectral characteristics (e.g. PSD) results are also stored as part ofthe signature. 50 h_a indicates the signature of spectralcharacteristics (for example the PSD) of frames 10 from thetransformation function in time at regions ROI_h. 50 h_b indicates thesignature of the spectral characteristics (e.g. PSD) of backgroundframes 10 bg from the transformation function in time at regions ROI_h.50 h c indicates the spatial spectral characteristics (e.g. spatial PSD)of one of the sequence frames (of sequence frames 10) at region ROI_h(which was also calculated by the processor). 50 h_d indicates thespatial spectral characteristics (e.g. spatial PSD) of one of thebackground frames of 10 bg at region ROI_h (which was also calculated bythe processor). The corresponding region ROI_h and corresponding vectorVec_h are also stored as part of the signature.

50 cb_a indicates the signature of spectral characteristics (for examplethe PSD) of frames 10 from the transformation function in time atregions ROI_cb. 50 cb_b indicates the signature of the spectralcharacteristics (e.g. PSD) of background frames 10 bg from thetransformation function in time at regions ROI_cb. 50 cb_c indicates thespatial spectral characteristics (e.g. spatial PSD) of one of thesequence frames (of sequence frames 10) at region ROI_cb (which was alsocalculated by the processor). 50 cb_d indicates the spatial spectralcharacteristics (e.g. spatial PSD) of one of the background frames of 10bg at region ROI_cb (which was also calculated by the processor). Thecorresponding region ROI_cb and corresponding vector Vec_cb are alsostored as part of the signature.

50 f a indicates the signature of the PSD of frames 10 from thetransformation function in time at regions ROI_f. 50 f_b indicates thesignature of the spectral characteristics (e.g. PSD) of backgroundframes 10 bg from the transformation function in time at regions ROI_f.50 f_c indicates the spatial PSD of one of the sequence frames (ofsequence frames 10) at region ROI_f (which was also calculated by theprocessor). 50 f d indicates the spatial spectral characteristics (e.g.spatial PSD) of one of the background frames of 10 bg at region ROI_f(which was also calculated by the processor). The corresponding regionROI_f and corresponding vector Vec_f are also stored as part of thesignature. For specific applications, the signature may comprise lessthan 12 sub signatures (e.g. only one, two, three, four or five of thesignatures in the signature 50).

Furthermore, the number of frames 50 n of frames 10 is stored in the inthe signature 50. This is for the purpose of future comparison—when thesignature is compared to a presently checked item—the same numbers offrames may be taken for the presently checked item for a better accuracyresult of the transformation to frequency range function. The comparisonwill be explained hereinafter in more detail.

Comparison with New Signature

According to a preferred embodiment, when the system operator wants tocompare a currently identified subject person with the signatures storedin the database it is performed as follows. For the new subject person,the method as explained hereinabove is performed until the step offinding the RIOs (not included), i.e. obtaining background, foreground,body portions, vectors, and background sequence (equivalent to 10 bg).

A. Geometry Adaptation to Find New ROIs

Then, the ROIs for the new subject person are obtained in relation tothe geometry of the currently checked data signature in the database.The new subject person ROIs are positioned in the spatial conditions (in3D) as close as possible to those of the currently checked signature.The ROIs are chosen at the same distance and spatial angle from thevectors (e.g. from the center of the vectors) as the distance andspatial angle between the signatured angles and distances from theirvectors. It should be clear that distance/angle of the head vector ofthe subject person being checked should be the same as thedistance/angle of the head vector in the signature, and the same goesfor the center body and feet features, mutatis mutandis. Thus the foundROIs are integrated into the sequence frames being checked (equivalentto sequence 10) and background sequence (equivalent to sequence 10 bg).

The distance and spatial angles between the signatured ROIs and thesignatured vectors are calculated by the processing means or optionallythey could be calculated and stored within each saved signature.

B. Compare Background ROIs and Prediction

Then, a transformation function to the frequency range in time isapplied (e.g. FFT) to the background sequence (equivalent to sequence 10bg) of the currently checked subject person—at the new ROIs (adaptedaccording to the signatured geometry), thus producing their spectralcharacteristics (e.g. PSD) at the ROIs—60 h_b, 60 cb_b and 60 f_b (forhead ROI, Center Body ROI and Feet ROI respectively).

The results of the background sequence frequency PSDs 60 h_b, 60 cb_band 60 f_b (relating to head, center body and feet respectively) of thecurrently checked subject person are inputted into a transfer functionmodule 75 along with the currently checked signature backgroundsignatures 50 h_b, 50 cb_b and 50 f_b, as shown in FIG. 10.

The transfer function module 75 evaluates the difference between thevalues of 60 h_b and 50 h_b (e.g. multiplication matrix, convolution,Fir filter, etc.). This relative difference is applied by a shiftingmodule 80 shifting the value of the currently checked signature originalframes time signature 50 h_a (in relation to the head) in a proportionalmanner (as the difference between 60 h_b and 50 h b) to obtain acorrected/predicted signature 50 h_a′. In the same manner the transferfunction module 75 evaluates the difference between the values of 60cb_b and 50 cb_b and evaluates the difference between 60 f_b and 50 f_b.These relative differences are applied by shifting module 80 shiftingthe value of the currently checked signature original frames timesignatures 50 cb_a and 50 f_a (in relation to the center body and feetrespectively) in a proportional manner obtaining corrected/predictedsignatures 50 cb_a′ and 50 f_a′ (in relation to the center body and feetrespectively) in the same manner with the necessary changes.

C. Comparison

Finally, the original sequence frames of the currently checked newsubject person are obtained. The processor carries out a transformationto the frequency domain (e.g. FFT) function in time to all the signalsin the frames at the new found ROI portions/locations, thus producingtheir frequency characteristics (e.g. PSDs)—60 h_a, 60 cb_a and 60 f_a(relating to head, center body, feet respectively).

These values are evaluated with the shifted values found of the shiftedsignatures (50 h_a′, 50 cb_a′ and 50 f_a′) respectively to find thecoherence between them. The coherence function of the values beingcompared produces a result indicating how close the subject persons (ofthe signature and the currently checked person) effect on the ROIs are,indicating that they are the same person. For example, a positive match(identification) would be if the coherence function would indicate uponan 80% or 90% similarity between the values. A threshold percentage ofsimilarity can be chosen by a system user wherein a percentage above thethreshold indicates a positive identification and a percentage below thethreshold indicates a negative identification.

The coherence is compared by a coherence function module 100 (shown inFIG. 11) between the frequency PSDs of 60 h_a and 50 h_a′. If thecoherence level is above a certain threshold then an identificationbetween the currently checked subject person and the signed person (thecurrently checked signature in the database) is deemed positive. If thecoherence level is beneath a certain threshold then an identificationbetween the currently checked subject person and the signed person (thecurrently checked signature in the database) is deemed negative. Asimilar comparison is made between the frequency PSDs of 60 cb_a and 50cb-_ab′ and between 60 f_a and 50 f_a′, mutatis mutandis.

A positive identification can be determined with only one of the three(head, center body, feet) comparisons being positive or with two beingpositive or optionally with all three being positive.

A similar determination may be made with the spatial frequency spectralcharacteristics of the signatured subject and the currently checkedsubject, wherein the adaptation is according to the spatial backgroundsignature frame 50 h_d, (and 50 cb_d and 50 f_d and also using 50 h_c,50 cb_c and 50 f_c), mutatis mutandis.

It should be clear that the present invention preferably comprises, whencomparing the features of two subjects, that the adaptation and shifting(ROIs and vectors) for the comparison can be carried out from the firstto the second and from the second to the first. This is true especiallywhen comparing two signatures, the operator may choose adapting thegeometry features of a desired signature to those of the other or viceversa.

However, the present invention also comprises comparison betweenspectral characteristics which one has not necessarily been adapted tothe other (e.g. by transfer function). The coherence is evaluated and anidentification is determined. Therefore, two signatures may be comparedin the coherence function module even without the same number ofsequence frames or same relation between ROI and vector. The spectralcharacteristics are just compared and a determination can be made.Naturally, the same amount of frames and corresponding ROIs in relationto the vectors may contribute to effectiveness of the comparison.

FIG. 12 illustrates an example of a coherence function module 100A. Twofrequency spectral characteristics (e.g. PSDs) A and B (e.g. twosignatures in a database, or (1) a currently determined (according tothe present invention method) spectral characteristics and (2) asignature in a database) as explained hereinabove are inputted into thecoherence function module 100A.

The coherence function module 100A comprises an error function module200 that obtains the error between the inputted two frequency spectralcharacteristics (e.g. subtracts one of the initial inputted frequencyspectral characteristics from the other initial one). If the errorbetween them is beneath a certain threshold—then a positiveidentification is determined between the persons of the frequencyspectral characteristics of A and B. If the error between them is abovea certain threshold then the error is transferred to an adaptive filter250 (e.g. BSS, LMS, RMS, other convergent evaluation modules) thatadapts the values of spectral characteristics A according to the errorlevel (e.g. adapts the coefficients of the filter according to the erroroutput level). The adapted spectral characteristics is inputted into theerror function module 200 that subtracts adapted A from B. If the errorbetween them is beneath a certain threshold—then a positiveidentification is determined between the persons of the frequencyspectral characteristics of A and B. If the error between them is abovea certain threshold then the error is transferred to adaptive filter 250that adapts the values of spectral characteristics A according to theerror, and so on and so forth.

If the loop continues a number of times above a predetermined thresholdthen the final determination of the identification between A and B isdeemed negative. If the loop continues a number of times beneath thepredetermined threshold then it means that at some point the errorcalculated is beneath a certain threshold and the loop is broken thusdetermining a positive identification.

The comparison may be made between time or spatial spectralcharacteristics as described hereinabove. Optionally, the first time thespectral characteristics of A is fed to the error function module 200the adaptive filter 250 may adapt it according to a predefined adaptingfunction. Optionally the first time A is fed to the error functionmodule 200—A is not adapted at all.

Optionally, the time of the convergence may be the threshold factor to apositive identification or not, i.e. if the loop breaks before apredetermined threshold time the identification is deemed positive.Optionally, when given 2 signatures and it is wanted to check which ofthem corresponds to a third signature, a possible manner of determininga positive determination is which of the two being compared to the thirdsignature with the corresponding error therebetween being converged in afaster manner.

The present invention system is adaptive, i.e. it can take multiplevideo samples of a certain subject surrounding and corrects itssignature features according to the feedback received from the latervideo samples. This improves the coherence (and credibility/reliability)of the signature. The final signature can be an average of the frequencyproperties of a few samples of the subject person.

According to an embodiment of the present invention, various types ofmachine learning methods may be used for classification of the savedsignatures.

The present invention can be used to efficiently and quickly search fora specific person in a video, on-line or during a post event analysis.For example, if security forces have a video sequence of a wantedsuspicious subject, they can obtain his surrounding ROIs frequencyfeatures according to the present invention and compare with othersubjects (in video camera films) surrounding ROI frequency features(optionally pre-signing them too) or with signatures saved in adatabase, to receive a positive/negative identification determination.

The present invention enables personnel to mark a moving subject personon a video for analyzing and signature as explained hereinabove and alsoenables an automatic marking, analysis and signature of subjectsentering a field of view (e.g. using appropriate tracking applications)and automatic comparison with the database. For example, if there is awanted suspicious subject and his signature is saved in a database, theautomatic feature may obtain the surrounding ROIs frequency features ofeach person entering a field of view of the cameras of the system and(optionally sign and) adapt and compare them to the signature of thewanted suspicious subject.

Tracking

When a suspicious subject enters one of the system cameras field of viewsecurity personnel can mark the suspicious subject on the viewing screencausing the operation of a tracking system (or a subject can be trackedautomatically). The tracking system application is an applicationsoftware generally on the same processing means that enables marking asubject (with a computer mouse or with touch screen, or automatically bymotion detection software etc.).

The present invention also enables continuous tracking of a subjectmoving through adjacent cameras fields of view. First the subject istracked within the first camera field of view. While in the first camerafield of view, the subject may be tracked by carrying out a signature toa sequence of frames and to a consecutive sequence of frames after apredefined time. The most newly-currently frames sequence being signedhave their ROIs/vectors geometry adapted to those of the previouslysigned frame sequence signed (or one of the previous ones) and thepositive/negative identification is determined. If the identification ispositive the tracking continues.

Preferably, a full signature with a full optimization of ROIs is carriedout for the most newly-currently frames sequence being signed, eventhough for the comparison/determination its ROIs/vectors geometry havebeen adapted to those of the previously compared signature.

If the identification is negative the tracking ceases. Preferably, thetracking does not cease, but subsequent sequence videos from cameraswith adjacent fields of view are analyzed (compared with the lastsignature in the field of view) to find the subject and continue thetracking if identification is deemed positive. The present invention isadvantageous as it allows continuous tracking from one camera field ofview to another, as the frequency data is analyzed in a centralprocessing means regardless of a specific camera field of view. Thepresent invention thus enables camera to camera re-identification andcomplete handshaking (as the sequences evaluated are deemed to bepositive i.e. of the same subject person) for continuous tracking. Alsothe tracking may be along multi-cameras and multi-platforms with therecognition seamlessly across zones.

The present invention may distinguish between multiple people in thefield of view, multiple foregrounds by using appropriate distinguishingapplications such as Blind Source Separation (BSS) or such as AdaptiveBeam Forming.

When the tracked person exits the camera field of view and then returnsto it (or to any system camera field of view), the tracking can resumeoptionally indicating that the person has returned and is once againbeing tracked. For this option sequences of system cameras are analyzedevery predefined time after the subject leaves the field of view.

Uses

Thus the present invention is very useful for use with mounted/mobilecameras with no special mounting required. The present invention has lowsensitivity for uneven lighting due to the luminance correction feature,general frequency analysis and shifting feature. The present inventiondoes not require a facial view or hair view.

There is no dependency on clothes, external appearance, change of wearor movement direction/orientation—the frequency analysis on thesurrounding ROIs of a moving person are clearly not substantiallyaffected by these changes. The present invention can perform a 360°detection with no need for a specific position towards the camera.Tagging anybody can be carried out anywhere. The signature can be madeindoor or outdoor. The identification can be made close to real time andwithout the cooperation of a subject person. The present invention isespecially efficient because several times a subject in a video isunidentifiable. The surrounding ROI frequency features can enable apositive identification.

The present invention is useful for a variety of applications. Thepresent invention can be used for homeland security and intelligenceindustries with a near real-time video analytics solution identifyingsuspects based on biometric ID.

The present invention may be used for personalized retailing (commercialanalysis) identifying previous customers and linking them with theirstored preferences for commercial use (e.g. identifying the same shopperat the cash register and analyzing his purchases), or connectingshoppers to a specific track through different shop departments, etc.The present invention may be used for Commercial Security andindustrial/healthcare facilities Safety. The present invention mayprovide video content management with structured repositories of videocontent in a searchable, intelligent manner, possibly being accessibleto the public.

While some of the embodiments of the invention have been described byway of illustration, it will be apparent that the invention can becarried into practice with many modifications, variations andadaptations, and with the use of numerous equivalents or alternativesolutions that are within the scope of a person skilled in the art,without departing from the spirit of the invention, or the scope of theclaims.

1. A method for generating a biometric signature of a subjectcomprising: obtaining a plurality of sequential video frame images of amoving subject from a video segment; obtaining a portion of each framecomprising a surrounding of the moving subject; carrying out atransformation function to the frequency domain on one or more of saidportions of the frames comprising a surrounding of the subject; andoptionally saving the spectral characteristics of said transformationfunction in a repository.
 2. The method according to claim 1, whereinobtaining a portion of each frame comprising a surrounding of the movingsubject comprises: obtaining foreground frames corresponding to theplurality of sequential video frames, each comprising the movingsubject; generating a vector representing the direction of the movingsubject; determining a Region Of Interest (ROI) at a position inrelation to said vector; and determining said portion of each framecomprising a surrounding of the moving subject, at a location on eachframe corresponding to the location of said determined ROI.
 3. Themethod according to claim 2, wherein obtaining foreground framescorresponding to the plurality of sequential video frames comprises:obtaining a background frame comprising the background of the pluralityof sequential video frame images; subtracting said background frame fromeach of the plurality of sequential video frames.
 4. The methodaccording to claim 2, wherein generating a vector representing thedirection of the moving subject comprises: obtaining body portions ofthe foreground objects of the foreground frames; obtaining referencepoint frames corresponding to the foreground frames, each comprising areference point about or at an edge of the corresponding location ofsaid body portions; combining all the reference points frames into athree-dimensional coordinate system frame comprising all the referencepoints being at their corresponding location on the three-dimensionalcoordinate system frame; determining a vector in the three-dimensionalcoordinate system frame according to a sequence of a number of referencepoints from the reference points, that produce the most stable vector.5. The method according to claim 2, wherein determining a Region OfInterest (ROI) at a position in relation to said vector comprises: a.obtaining a plurality of background sequential video frames being asequence of frames comprising the background of the plurality ofsequential video frame images; b. integrating the vector determined inin each background frame; c. determining an initial ROI on eachbackground frame being at a predetermined corresponding position fromsaid vector; d. carrying out a transformation function to the frequencydomain in time to the ROI portions of said background frames obtainingspectral characteristics and determining the stability frequencies fromsaid spectral characteristics; e. integrating each of the plurality ofsequential video frame images with said initial ROI determined andcarrying out a transformation function to the frequency domain in timeto the initial ROI portion of said sequential video frames thusobtaining spectral characteristics and storing the sensitivity of saidstability frequencies of said spectral characteristics of the initialROI; f. shifting the initial ROI to a surrounding area in each of theplurality of sequential video frame images, and carrying out atransformation function to the frequency domain in time to the currentlyshifted ROI portion of said sequential video frames thus obtainingspectral characteristics and storing the sensitivity of said stabilityfrequencies of said spectral characteristics of the currently shiftedROI; g. repeating step f according to a predetermined shifting rule; h.after step f has been repeated for all sequences of the shifting ruledetermining the ROI of the initial and shifted ROIs with the highestsensitivity stored as the ROI.
 6. The method according to claim 4,wherein the body portions are one or more of: the head portion; thecenter body portion; the feet portion.
 7. The method according to claim1 further comprising a step of identification by comparing the obtainedspectral characteristics of the transformation function with spectralcharacteristics saved in a database, wherein an identification result isdeemed to be positive when the coherence level between both comparedfrequency spectral characteristics is above a certain threshold.
 8. Themethod according to claim 5 comprising performing a signature for asubject by obtaining and saving the vector generated and thecorresponding ROI portion determined, and further obtaining and savingone or more of the following items in a database in relation to saidsubject: the spectral characteristics of a transformation function tothe frequency domain in time to the ROI portions of the sequentialframes; the spectral characteristics of a transformation function to thefrequency domain in time to the ROI portions of the background frames;the spatial spectral characteristics of a transformation function to thefrequency domain at the ROI portion of one of the sequential frames; thespatial spectral characteristics of a transformation function to thefrequency domain at the ROI portion of one of the background frames. 9.The method according to claim 2, further comprising a step ofidentification; providing a signature of a subject person stored in adata base comprising a vector, an ROI, spectral characteristics of atransformation function to the frequency domain in time of a framesequence, spectral characteristics of a transformation function to thefrequency domain in time of background frames; wherein the Region OfInterest (ROI) at a position in relation to the vector is determinedsuch that it is in the same position in relation to the vector as thesignature ROI in relation to the signature vector; wherein said methodfurther comprises: i. obtaining the spectral characteristics of atransformation function to the frequency domain in time to the ROIportions of the background frames; ii. obtaining a relative differenceby inputting the spectral characteristics of a transformation functionto the frequency domain in time of the background frames of step (i) andsaid signature spectral characteristics of a transformation function tothe frequency domain in time of background frames, into a transferfunction; iii. obtaining the spectral characteristics of atransformation function to the frequency domain in time to the ROIportions of the sequence frames and shifting the value of said spectralcharacteristics of a transformation function to the frequency domain intime to the ROI portions of the sequence frames, in a proportionalmanner as said relative difference; iv. comparing the shifted values tothe signature spectral characteristics of a transformation function tothe frequency domain in time of a frame sequence; wherein anidentification result is deemed to be positive when the coherence levelbetween the compared spectral characteristics of step (iv) is above apredefined threshold.
 10. The method according to claim 1 furthercomprising an identification, said method further comprising: obtainingan error value between the spectral characteristics of thetransformation function and spectral characteristics saved in adatabase; wherein an identification is deemed positive if one of thefollowing conditions are met: I) the error value is beneath a predefinedthreshold value; II) the following consecutive steps are carried outless times than a predetermined threshold number: i. transferring theerror value to an adaptive filter that adapts the values of one of thespectral characteristics according to the error value; ii. obtaining anerror value between the adapted spectral characteristics value and theother spectral characteristics; iii. determining if the error value ofstep (ii) is beneath said threshold value; iv. returning to step (i)when the determination of the error value of step (iii) is deemednegative.
 11. The method according to claim 1 further comprising anidentification, said method further comprising: obtaining an error valuebetween the spectral characteristics of the transformation function andspectral characteristics saved in a database; wherein an identificationis deemed positive if one of the following conditions are met: I) theerror value is beneath a predefined threshold value; II) the followingconsecutive steps with possible recursion are fully carried out during atime duration less than a predefined threshold time: i. transferring theerror value to an adaptive filter that adapts the values of one of thespectral characteristics according to the error value; ii. obtaining anerror value between the adapted spectral characteristics value and theother spectral characteristics; iii. determining if the error value ofstep (ii) is beneath said threshold value; iv. returning to step (i)when the determination of the error value of step (iii) is deemednegative.
 12. A system comprising one or more cameras connected toprocessing means, wherein the processing means comprise: A) a database;B) a transformation to frequency domain module; C) a comparing frequencycoherence function module; wherein the processing means are configuredto generate a biometric signature of a subject comprising the steps of:obtaining a plurality of sequential video frame images of a movingsubject from a video segment; obtaining a portion of each framecomprising a surrounding of the moving subject; carrying out atransformation function to the frequency domain on one or more of saidportions of the frames comprising a surrounding of the subject; andoptionally saving the spectral characteristics of said transformationfunction in a repository.